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ABSTRACT 

A savple of 22 multiple-choice questions Is provided 
to point up the potentialities of thle type question, as %#ell as to 
dispel the nyth that "objective** tests require no thought, insight, 
or understanding* The questions have been taken froa a variety of 
tests constructed by Educational Teating Service, and are presented 
as they appeared in the original tests, i^ith the directions, when 
given, as they %^rQ given to the students* Each question is followed 
by a statistical analysis of its performance and by a brief 
discussion of the thought processes involved in formlatlng the 
questions and arriving at the correct ans%#er« fCB) 
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Tho piir;M)so.of this chiso lr)ol< at a group of mii! t i pi o -choice 
(j nest ions is tfi disprl a myJh: the myth of (he niiiitipio- 
< lioicp question ns n siiporficial oxo^ise — one that requires 
little thouj^ht, loss insi^^ht, and no understandinK. Like 
(ft her myth^, this one may be baser i on a shadowy memory 
of the fjast. })ut it hears little relation to present reality. 
"Wb.it is often overlooked . . . writes rJerorne Bruner in 
The Process of Ednvation. "is thai examinations can also 
ho allies in the buttle to improve curricula and teach inj?. 
Whether an examination is of the 'objective' type involvinp 
multiple choices or of the essay type, it can bo devised so 
as to empluTsize an understanding of the broad principles 
of a sul).iect. Indeed, even when one examines detailed 
knowledge, it can be done in such a way as to require 
understanding by the stuflenl of the connectedness between 
specific facts." 

Educational testing in its present form and prevalence 
is a response to the needs of contemporary American edu- 
cation. Tests of scholastic aptitude and achievement seek 
to measure certain abilities tliat eflucators want evaluatefi. 
At Educational Testing Service, this measurement is accom- 
plished hy leaving to the educators the responsibility for 
determining the (objectives of each test and by intimately 
involving them in the actual eonstriiction of test questions. 
The great majority of tests pulilished by ETS are prepared 
hy committees of school and college teachers. The com- 
mittees are selected to represent difTerent institutions, 
regions, and points of view, as well as the breadth and 
depth of scholarship neeried to prepare a test in a i)articu- 
lar snibjoct. In a typical year, over 500 individuals serve 
on such committees. 

If a test is to have relevance to education, a test com- 
mittee, before writing a single question, must consider the 
goals of instruction that are most important to measure. 
Most often this step results in the preptTration of a written 
statement or outline of what is to be tested. Without such 
forethought a test may measure very precisely some very 
unimportant things. If tests are to bo educationally useful, 
the test questions must demand more from the student than 
rote recall. The preparation of written test specifications 
gives each committee the opportunity to identify the knowl- 
edge and the .skills that ought to be mea.sured hy the test 
questions, and to lay s|>ecific plans for the preparation of 
such questions. 

The actual creation of the test questions is itself a 
crucial, arduous process. It is crucial because the loftiest 
of objectives are meaningle.ss if the questions in a . test do 
not require the .students taking the test to demonstrate 
attainment of those objectives. It is arduous because of the 
.several important steps that must be successfully com- 
pleted in the writing of the questions. 

First, it is essential that the author of a test que.stion 
identify and define the skill or knowledge he wishes to 
measure. He must satisfy himself that, among the many 
questions that may be asked, a particular question is worth 
asking. Next, he must devise a way of asking the question 



tliat will insure that the correct answer is not Identified 
in some superficial manner. As may be apparent in the 
examples which follow, this consideration requires preci- 
sion and artistry in the writing of test questions. A well 
conceived and constructed multiple-choice question should 
rr(iuire a student to select, weigh, and apply what he knows 
in order to answer the question correctly. 

Also, the author of the question must word the question 
nnd the answer clioices so that those who indeefl know 
what is being asked can answer correctly and those who 
are less knowing will, in effect, be defeated by their own 
ignorance. The art of writing a multiple-choice question 
requires, of course, a knowledge of the subject ami a 
knowleflge of how the subject is taught at the educational 
level for which the question is intended. Moreover, th* 
writer of the question should be familiar not only with 
wli.it students ought to have learned, hut also with what 
they ought not to have learned. That is, he should be able 
to construct each question in such a way that students who 
have mi,sin formation or misconceptions will be differen- 
tiated from those who are knowledgeable. Then each an- 
swer choice in a multiple-choice question will contribute to 
differentiating between competent and less-than-competent 
students. Statistical analyses of the sample questions that 
foHow will show how this differentiating power of a ques- 
tion flepends on the extent to which the wrong answers 
attract the less competent students. Questions can dis- 
criminate at different levels of ability. In effect, a question's 
difficulty and discriminating power depend on the sophisti- 
cation required to distinguish among the answer choices. 

To create questions that are important, appropriate, ac- 
curate, and discriminating requires the imaginative efforts 
of well-trained, experienced Inrlividuals working in concert. 
Typically, a test committee assumes responsibility not only 
for planning a test but for writing and reviewing the test 
questions as well. Usually each committee member writes 
a certain number of questions and reviews those .submitteti 
by his (xil leagues. An ETS test <levelopment specialist 
assists the committee in planning the questions, offers ad- 
vice on the construction of the questions, and reviews the 
questions as a technical editor. Finally, the committee 
selects from among all the questions written those that it 
deems of sufficiently high quality to use in its test 

After passing the test of expert judgment, questions must 
then pass a second test — the test of use. Is the question 
easy — |)erhaps too easy — or diflfieult — maybe too (Hfficult? 
Does it differentiate between able and less able students? 
Is it clear and unambiguous? Even the most competent 
teachers and scholars may occasionally misjudge such 
matters. If a test is to provide reliable measurement, pos- 
sible sources of error are best identified and eliminated in 
advance. Therefore, whenever feasible, questions are tried 
out exi^eri men tally before inclusion in a test. From such 
**pretesting" considerable information of value is derived. 
Each question in this booklet is followed by a statistical 
analysis that summarizes the kind of information obtained 
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nhoiii the pcrfnrmanro. ol Jhoso shjrlonts icMvd willi Ihc 
qiirslion hcfurr its actujil use in a <ost as well as after it,s 
achial use. 

The <Un]eu]ty of each quesHon may hp reaflily dotcr- 
minofl. Diflinilly is roprescntcd by <ho prrconlnfio of stii- 
Hrnts who answrr a qiiostion corrrclly: thr hi^^licr the 
pnroniape corrcrt. Hie fas-irr /he qucKlion. Similarly. <ho 
attracfivrnrss of oaoh answer choiee may he H(»trrmine(i by 
ihv niimhor of stihlrnts who scaled \\, 

Tho power of a question to rfifTcrcnt late amonj: sluflents 
of flifT(»reiit arademie abilities is a move eomplex iiiatler. To 
(Ictermino tliis, it is necessary to have for eadi stuclent a 
measure of tfie ability tbaf is f)ein^^ tested by tho question; 
this mi-asure serves as a criterion. As eirciilar as it may at 
firs* seem, one sucli readily available criterion is the stu- 
dent's score on the total test of which the question is a 
fiart. The validity of such a technique has l)een confirmed 
hy comparing the results of analysis apainst an internal 
criterion with those obtainefl with a Icjss readily available 
tndepemlent criterion. In essence. althoug)i a committee of 
experts may err in the formulation of a particular question, 
we may assume that its jud foment of what constitutes 
achievement in an area is generally valid. Hence the rela- 
tion between shidents' success on a question and success 
on the entire test of which the question is a part is an 
ajipropriate measure of the power of the question to dis- 
criminate between more alile and less able students. 

Rut since much of the power of a multiple-clioice ques- 
tion depends on the nature of the answer choices, obtaining 
measures solely of the quest ion'j^ over-all performance is 
not sufHcienl. Tf the strenj:tbs and weaknesses of a question 
are to be diagnosed, knowledge of flie performance of each 
of its strategic parts must also be obtained. Therefore, for 
ea<'h question an analysis is made of the ability of the 
group of students selecting each answer clioice. It seems 
reasonable to expect that, in general, those who are more 
able should select the correct response to a question and 
that, in general, those who are less able should more fre- 
quently find wrong answers attractive. To analyze the per- 
formance of each question, students may be divided (as 
illu.sf.rated in the following pages) into equal fifths based 
on their performance on a relevant criterion — usually total 
test score. When the answer choices in a question are efFec- 
five, more students in the highest fifth than in the lowest 
fifth will select the correct response, and more in the lowest 
fifth than in the highest fifth will select the wrong answers. 
Also, the number of students selecting the correct reSfXJnse 
will generally increase in each higher fifth, while the num- 
ber choosing the wrong responses will increase in each 
lower fifth. Inforniation of this kind may be userl by a test 
committee to select for its te^st only those questions which 
have proven to discriminate ajipropriately, or which, given 
the information, may be revised to make them sufficiently 
di.scriminating. The questions appearing in this booklet 
have all passed through such a screening. 

Despite aU thesr precautions, not every question pre- 
pared hy the niethods described' turns out to be a good one. 
Even after the most careful scrutiny of an expert com- 
mittee as well as a painstaking experimental screening, 
some do not measure adequately an important instructional 
goal, and some may contain errors of fact or interpretation. 
These, however, are the rare exceptions. 

The questions which appear on the following pages have 
been taken from a variety of tests constructed by Educa- 




tional Te^iting Servii'o. Many shnilar (luestions are now in 
use. Those presented hero are as they appeared in the 
original tests, with the directions, when given, as they were 
given to the sturl(»nts. Each question is followed by a stntia- 
tical analysis of its performance and by a brief discussion 
of the thoufjht fjrocosses involved in formulating (iie ques- 
tion anfl ill arriving at the corr<>ct answer, 

Stufly of this representative sample of multiiile-choico 
cluestions should loafi to a clearer understanding of their 
potentialities, and should help to dispel the myth that 
•'objective" tests require no thought, insight, or under- 
stamling, 



Question 1. 




The shading on the above map is used to indicate 

(A) population density 

(B) percentage of total labor force in agriculture 

(C) per capita income 

(D) death rate per thousand of population 
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Cwr rein t ion between sul-cc8h on this quest i'f>n and total acoro on test .47 



♦Correct answer 

In many of the multipie-choice questions included in tests 
in the social sciences, an attempt is made to require the 
student to make use of his general background of knowl- 
edge in the interpretation of materials. Thus, this question 
does not simply ask: What areas of the world have the 
highest population densities? Rather, it presents a novel 
situation in which the student must infer that, of the 
choices offered, only population density provides a plausible 
explanation of the shadings on the map. 



This qucsfion was nnsworor! corrrcily hy Jilmnsi flO per 
cont of n firnwp of cf)ll('Mo soninrs. Mowovrr. alllioiifih h\r^o. 
numhors of \nwor af)jlily sfiirlc nls woro sut'C'ossfiil nn if. the 
qurslion dirl difTcronlinlo hfhvern some of tht- very pnnrost 
s(ii(l('n(.s niul iho rost of thnsp tcsltMi. 

An f'xnmination of Ww ninp t'lonrly shows ihoi choice 
(A), /)o;>iilation rlonsi/y, is /Jrc proper rospoM.so. The flnrk- 
f'sf. sliMfliiij(, which nrcorrlin^i fo Iho mnp's lo?;cnfl inclicnfos 
fhr hiphrsf clcirror of whnfpvor (lio sli.'iilin^ roproscnfs. 
rovrrs such h'l^U popiilafinn rJrnsity aroas /us Ihc north- 
oasforii parf of lhc» UniloH Sfafos, a hirpo pari of Eiiropc. 
Uir Nilp vnlloy, Inrlia, -lapan, ninl EastfTn China. If (his 
woro not a sufTirionf rhir. iho arras with fho li?:h(osf sharl- 
inp iiu-hifio such iiiiflorpopnlatod aroas as fhf Arrlir 
rojiioMs, tropical South Amcrit-a. tho Sahara anrl A rahian 
fJoscrts. aiif! most of Australia. 

Choice (R), the j)rrc(>n(af:e of tola! lahor force in a^'ri- 
ciiKiire, while possihiy aitraetive if only Tnrlia anH China 
are examined, is clearly incorrect when ai^piieH to the 
Northeastern United States. This t hnice was taken hy 10 
per cent of the least ahle proijp. !)iit was not allraetivo to 
any of the other groups. 

Choice (C). per caj)ila incfinie. attracted morlerately 
tho.se in the lowest (wo-fifths of the j^roup. Per (\npita 
income coiifrl he plansihie only if ihr stiulenCs analysis of 
the map took in solely the dark shadin^j in the Uniteri 
States anrl Western Europe, anrl e\-en lliere it is n<:)f 
entirely correct, hiit the dark .sharlinp: in China anrl Tnrlia 
could c<»rtainly not inrlicatr hi^^di per cai)ita in(X)me. The 
reverse observation is true of chf>icc (D). rioatli rale ]ier 
(honsatui of [)Of)Mlaf ion. since (he latter inijrht he expected 
to h(» hij?h for Fnrlia anrl China, hut low for the United 
.States and Europe. Only nine students Sf»lectef| this choire. 



Question 2 
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The ^raph above ropvcseiits the political cnmposition from 
1922 to 1950 of which of the folHiwiii«? 

(A) German Bundestag; 

(li) French National Assembly 

(C) Italian Chamber of Deputies 

(D) British House of Commons 
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To answer this f|uestion {•orrertly. the student musi he ai)le 
to do several thinj^s. First, he must he ahle to read the 
Kraijh. Then, usinji; the information he can infer from it, he 
must interpret it in tho li^jlit of liis knowlml^e of European 
historv and Kf>vernment froni 1022 in ]9r>r) and draw a con- 
elusion concern in which legislative body may properly be 
so depicterl. In such a process, it. is possible for difl'erent, 
stiuients to make use of dilTerent information to arrive at 
the correct answer. 

Fn examining the <:ra[)h. (he student shoulrl note that tlie 
party system shown is essentially a two-party one, althouf,di 
(here is a third party tliat. for most of (he period shown. 
flecrease<l in representation. He may also note the years in 
which elections were helrl. the years that Party A received 
majorities, the fact that Party B difl not receive a majority 
until 1945. anrl the fact that in the ele('tions of 1923 and 
1929 neither party receiverl a majority. 

In considering the first of the four possible answers, 
the German Bundestap:, the student should recognize that 
during the period through 1932. that of the Weimar Repub- 
lic, no single party in Germany was able to attain a 
majority, partly because of the multiplicity of parties. After 
Hitler came to power in 1933. jiarties other than the Nazi 
Party were oullawerh These facts rlo not fit the graph, as 
most of the students recognized. 

The second possible answer, the French National Assem- 
bly, contained far more than three parties both before and 
after World Wur II. In spite of this, many students. es]>e- 
cially those in the lowest two-fifths, found this choice 
attractive. Only three of the highest group selected it, 
however. T*he Italian Chamber of Deputies should also be 
rejectcfl as a possible answer because, after Mussolini came 
to power in 1924, it became le.ss and less importatit until, 
in 1938, it was suiierserlerl by a Chamber of Fasci and Cor- 
porations and political parties were suppressed. This choice 
atfracterl only five nr six students in each of the fifths. 

Over half of the group of college students who attempted 
this (luestion selected the correct response, the British 
House of Commons. Party A. on the graph, corresponds to 
the Conservative Party, Party B to the Labor Party, and 
Party C to the Liberal Par*y. The large Conservative 
majorities of 1924 and 1931 are clearly shown, as is the 
attainment of a majority by the Laboriles in 1945. 



ERIC 



Question 3 

**lfi a fliiHh it eiiinc upun mo that thorr was ihr rt'jiHon for 
ndvttiicinK poverty with ndviinein^ woiiUh. With the ^i-owth 
of poputntton, lan<\ i^rown in value*, hmH thv rnvn who work 
it fiiiist pay more for th4» priviU'^c. In allowinj; out' man lo 
own the Itmcl (ni which aticl from which othor m<;n live, wo 
have m«(l« them hiH hondsnien in a tk»^rfO whit h increanoH 
»M material proj;re.sj» jioe.s on. This i.H tlie suhtk' a)chom>' 
that in ways they do not roalixe is cxtruotifi^ from the 
masNen in every eivili/.od covin try the fruits of their weary 
toil/' 

The porHOn moMt likely to have written iUvsi* wortis in 

(A) John Jaeob AHtor 

(B) William Jonnin^iH Bryan 

(C) ThorMtein Veblen 

(D) Mneoln StelTenH 
(K) Henry Cleorgo 
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* Cor reel nnawor 



This question presents the student with a quotation .Tn<i 
iLsks him to identify the i)orson most likely to have made 
the statement. Note that the student is not asked to. or 
expected to, recognize the statement from riiemory, Tn.stead, 
he is expeeted to read the excerpt carefully, to evaluate 
it in terms of his knowledge and understanding of Ameri- 
can intellectual history, and then to select from the five 
names listed the person to whom the statt^ment might most 
reasonably he attributed. 

Some interesting results; may he seen by examining the 
statistics obtained when the question was administered in 
a college-level American history test to a group of very 
able high school seniors. Because the question was a difTi- 
cult one for the groups many students cho.se not to answer 
it at all rather than risk being penalized for an incorrect 
guess. This helped lo difi'erentiate between, the able stu- 
dents and the less able on(;s. Whereas 32 of the students in 
the lowest fifth omitted the question, only four of the 
students in the top fifth did so. 

Choice (A) served to attract only a few of the less able 
students. None of the top ;-rroup thought that the statement 
could have been made by John Jacob Astor. An entre- 
preneur of the early nin(;teenth century wJ o built up a 
vast fur trading empire in the Pacific Northwest, Astor 
would most likely be associated with the opening up of tbe 
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Ori';zon ("ounjry. In this hunv, i)nsi»ltle(! reuion whore land 
was tluMi readily available, he would liarflls' have h(^t*n r<in- 
rvvi 'd Willi the particular economic prolilciii disi'nss(Ml in 
the fpiolniiiiu. Moi'eover. anyone wlio recngni/es the tiame 
has n»i reason to a^sociat(» it with reform. 

CJioire (B) is a more .soph i-^tica ted wrong an.swer than 
is (A). Th(» students who selected Wiliiani Jennings Hryan 
as the probable author of tbe passajze \vi»re un(hiuht(»(Hy 
awar(» of Rryans reputation as "^Phc^ Great Conuuonor" — 
the sr»!f-styled def(»ud(T of tlie p'-ople against the "dictator- 
ship" of Wall Street, (heir ('hampior. in the campaign for 
free coinage of silver Although the last senlenc(» of the 
quota! itni might he slightly reminiscent of Hryan's styl(». 
the particular ideas are not those exploited by an orator 
who made free silver a "cause celehre " Tfiis nnsw(^r, while 
attractive to some of (tie students in the lower two-fifths, 
was mt>st jjopular with those in the middle fifth. This may 
he partly explained liy its greater sophisticati(m, hut the 
fact that most of the lower grou]) had already chosen to 
omit the question probably accounts tor the failure of that 
^r(»up to choose (R) . 

Choice (C). Thorstein V(»hlen. served to separate the 
better students from tbe very best. Few of the poorest stu- 
dents chose Veblen. partly because of the reasons men- 
tioned above, and po.ssihly also because they may not have 
been familiar with Veblen. Tbe brighter students may have 
been more familiar with his name and ideas or at least 
with (he fact that he was a critic of the American economic 
.system at the turn of the century. They (li<l not know 
enough, however, to realize that Veblen was not concerned 
with the monopoly of land as a cause of poverty. 

Very few of the top group were attracted to choice (D), 
Lincoln StefTons- Concerned primarily with the problems 
o( the cities, Steffens is perhaiw best known for his expos- 
ure of munici|)al corrui)tton. 

Henry George, the correct response, was selecte<l by over 
half of those in tbe highest fifth, but only six of the lowest 
group chose this correct answer. Altogether on!}' 2ft per 
cent of the total group could answer the question correctly, 
for a thoughtful understanding of tbe ideas of Henry 
George was required. The statement di<l not contain any 
of the catchwords usually superficially connected with 
(jeorge. such as "single tax" or "unearned increment.'* 



Question 4 

In the following qucHtions you are asked to make inferences 
from the data which arc given you on the map of the imag- 
inary country, Serendip. The answers in most instances 
must be probabilities rather than certainties . The relative 
size of towns and cities is not shown. To assist you in the 
location of the places mentioned in the questionSf the map 
is divided into squares lettered vertically from A to E and 
numbered horizontally from 1 to 5, 
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Which of the following cities would be the best location for 
a fitcel mill? 



(A) Li 

(B) Um 

(C) Cot 

(D) Diibc 

.^tafiMtn-nl A 'in! \- sis 



RESPONSKS 

On)it 
A 

c; 

D 
To-al 



(3A) 
(3B) 
(3D) 
(4B) 



Studt-nls Cl.-issifu'd by Totnl Test Score 

LOWEST HIKMKST 



8 

10 
40 
4 
9 
71 



2 

84 
1 
6 

93 



!*er <'ent of total group of liTO students aoswcrinp correctly . . . . T.^)*^ 
('orrelalion l>i'twi»un sut uess on this tiucHtM)ii and total K<.()r('<)n li'Ml . . . -l*! 

».\ Hornuwhat diffi-reiU form of iinalysiH was used for this test. I )is<'rei>an<'ies 
in total tuirnl»(»rs in ea<'h ^roup ar<' e.-nisetl l>y drop'out «)f sludents; not I'orn- 
plciinK the l(st .) 



♦Correct answ er 



The (luestion re|)r(«lurt»<l henealli the map requires 
know letl^o of the natural resources used in [)r()(hjriim steel 
and an awareness of the im|)t)rtan<'e of transp(>rtatioii faelli' 
ties in l)rin«;in^( these rosourees to^etht^r. If was part of n 
Konoral achieve meat tost ^Mveti to h\yM sehool seniors. 

I'lie stiidtMit wlio knows that iron is the hasi<' raw mate- 
rial of steel and that eoal eomnionly |)ro\ ides the ntvcssnry 
source of heat would proceed to locate deposits of these 
resources in relation to th(» cities listed in the (juestion. 
lie would h(» ahle to eliminate Cot immediately. sinc4> there 
is no iron or coal in its \'icinity. although Cot miglit he an 
attractive cfioice to students wiio mistakenly think that 
copper is a l)asic ingredient (»f steel. Both Li and Duhe are 
locattnl reasimahly near supplies of iron, and therefore 
miKld he attractive rh(»ice.-*. Um. however, is the more 
clearly "correct" response, heeause not only are deposits 
(^f ir(»n i\n(\ coal nea»'hy. hut they are more readily trans- 
portahle hy direct railroarl routes. 



Question 5 




A map of an ima^jinary country, such as that shown above, 
offers numerous possibilities for (luestions which measure 
important understanding's. One could ask several (|uestions 
requirinf: an \inderstanding of the symbols used on the 
map. To determine student comprehension of the meaning 
of contour lines, for example, one mi^ht ask which railroad 
h.as the steepest grades to climb. Similar questions can he 
developed »vhicli require knowledjre of tfie factors influenc- 
ing population distribution, economic activities and so on. 



In which of the follow in j^enturioH was the piece of sculp- 
ture shown above most pi^^ably produced? 

(A) The fifth century B,C. 

(B) The fourteenth century A«D« 

(C) The sixteenth century A,D, 

(D) The eighteenth century A,D, 
(K) The twentieth century A.D. 
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♦CorrcMM answer 



Tliis (iiio«tion on art appeared in a lost of ^'rncrnl hnrk- 
j^roiind ^Mven to college seniors and ^'raduato students. To 
answer llio question, the student must apply Iiis knowled^'e 
of tlie characteristics of various periods in the history of 
sculpture in order to place the statue within its proper 
period. 

The statue can be identified immediately as a product 
of western civilization because of its subject matter an(l 
design. The sculpture most commonly associated with 
western art in the fifth century B.C. is that done in Greece. 
During' that period Greek sculpture showed its character- 
istic idealization of the human figure. If the student con-, 
trasts a well-known statue from that period, such as 
Myron's 'Discobolus" ("The Discus Thrower") with the 
statue in the picture, he can easily discover that the treat- 
ment accorded the two figures is not the same. The figure 
of the woman pictured is not idealized, and although the 
frontal pose is found in Greek sculpture, a turned head on 
the figure is not. Therefore, choice (A) can hardly be 
correct. 

In some countries sculptors of the fourteenth century 
A.D. began to create statues for use in palaces, chapels, 
and tombs of private individuals, but their work retained 
the qualities of the representations of saints found in the 
Gothic cathedrals of Europe. Statues of men and women 
associated with Christian history were also used as archi- 
tectural decorations. In the fourteenth century probably 
only Eve woulfl have been carved as a nude female figure, 
and she would certainly have been portrayed in a more 
modest pose and with a fig leaf. Although, like the figure 
of the woman pictured, the figures of this period of Chris- 
tian art are not idealized, the statue of the woman displays 
none of the other characteristics of medieval art, and 
choice (B) can be eliminated. 

The fifteenth century saw the beginnings of the Italian 
Renaissance, and by tiie sixteenth century sculptors were 
again presenting idealizations of the human figure, most 
often in a Christian rather than classical context. The 
statue pictured above shows none of the heroic character 
that is found in the works of Michelangelo and none of the 
exaggerated stylization that appeared in the works cf his 
followers. Thus, choice (C) can be regarded as inaccurate. 
Early in the eighteenth century sculptors frequently dis- 
played a self-eonscious coy femininity in their female 



mides. but by the end of thni century sculptors sucli as 
Canova had adopted thp classical ideals and imitated 
classical n\o(U^ls. C'nnsctiuently, choice (D) is not a valid 
one. for the statue in the photograph does not resemble an 
eighteenth-century interpretation of a Greek goddess or 
nymph. It does, however, realistically present a hunuin 
figure without any attempt at idealiwition. The realism 
here is thoroughly modern. The use the sculptor makes of 
texture points to the modern period, for it was only late 
in the nineteenth century that sculpture (sucli as Renoir's 
•"riie Washerwoman") used nonrepresentational texture as 
an integral part of the design and aesthetic effect of the 
work of art. Minor details in the statue such as the shape 
of the nixse, the lialf-dosed eyes (instead of o|)en sockets), 
ami the line of the hair indicnte that the work is not clas- 
sical in origin, and at the same time they point to the 
twentieth century school of art. The statue is actually 
"Jungc Frau." the work of the modern German sculptor, 
Gcorg Kolbe, 

The statistics show that the? question was an easy one 
for the particular group that took the test. Tlie correct 
an.sw/^r was chosen by 02 per cent of the students. However, 
more of the students in the higliest group than in the lowest 
group selected the correct response, and the incorrect 
answers were generally more apiiealing to the poorer stu- 
dents thnn they were to the better students. 



Question 6 

In the following ciucHtion you are given a complete sentenuu 
to be rephrased aeeording to the directions whieh follow it. 
You should rephrase the sentence mentally to save time, 
although you may make notes in your test hook if you wish. 

Below the sentence nnd its directions arc listed words or 
phrases that may occur in your revised sentence. When you 
have thought out a good sentence, find in the choices A to K 
the word or entire phrase that is included in your revised 
sentence. The word or phrase you choose should be the most 
accurate and most nearly complete of all the choices given. 

Although the directions may require you to change the 
relationship between parts of the sentence or to make 
slight changes in meaning in other ways, make only those 
changes that the directions require ; that is, keep the mean- 
ing the same, or as nearly as the directions permit. If you 
think that more than one good sentence can be made ac- 
cording to the directions, select the sentence that is most 
exact, effective, and natural in phrasing and construction. 

Sentence: .)ohn, shy as he was of girls, still managed to 
marry one of the most desirable of them. 

Directions : Substitute John's shyness for John, shy. 

Your rewritten sentence will contain which of the following? 

(A) him being married to 

(U) himself married to 

(C) him from marrying 

(D) was himself married to 

(E) him to have married 
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♦fjorrcci anawer 



In flovolopinj? this <ype of qiirsHon the cnmmittco of exnm- 
inrrs rcasoncMl ihni mast ^ood writers rtvonstnic^ senlencf.'S 
to chan^'e omphcisis. to improve stylo, to avoifl amhi^niity, 
<ir to oliniiiiate vorh(>sity. and thnt ihc f^ood student should 
ho able to mako such chaii^* withoi: involving himsrif in 
structuru faults or ^iranimatical errors. This type of ques- 
tion is dosij^nod, therefore, to assess the student's miistery 
of variety in sentence structure, his abihty to make a 
chMnfje within a .sentence so that it says what he intends 
to say more smoothly, concisely, and efl'octively than the 
ori^iinal version may. Furtliermore, the committee <lecided 
that to test this ability the question should require the 
student first to construct his new sentence mentally and 
then to compare his answer with a number of iwssible 
answers presented to him. As choices for answers they 
chose the kinds of expressions wl>ich slurlents inciufle in 
their own writlnj; when they attempt to solve problems of 
sentence variety ami become enmeslied in K^f^n'matical 
incongruities or verbal obscurities. 

In order to select clioico (A), a sentence ]iko John.s 
shyne.ss with girls did not stop him bcin^? married to the 
most desi rable of thcni would have to be used. To make 
this sentence correct, forma! written English demands that 
the worfl being be preeerled by the possessive pronoun his. 
Choice (B) presents a sentence similar to Despite John's 
shyness with girls, he managed to get himself married to 
one of the _most desirable of them . This sentence is wordy 
and inapi.ropriate in its tone (for formal English). Choice 
(C), however, yields a sentence on (he order of John's 
shy^iK' ss with girls did not prevent him from marrying one 
of the most drsirablo of t hem. This ret;nn.s iho monninyr »f 
the original sentence and contains no errors in grammar; 
it is the correct answer. The fourth choice might lead to 
John's shyness with girls did not keep him single; he was 
himself married to one of the most desirahJc of them . This 
sentence changes the meaning of the original sentence, and 
it is. at the same time, ambiguous in its own meaning. 
John s shyness with girts was not a reason for him to have 



married the most desirable of them — 



attempt to use the fifth choice, r'esults in 
of meaning. It is therefore unacc eptable 
grammatically correct. 

This question was answered correctly 
the college-bound high school students 
Most of tile students in the two highest 
correctly, while a majority of students 
groups answered the question incorrectly, 



this sentence, an 
a complete change 
, even though it is 

by 61 per cent of 
who attempted it. 
fifths answeretl it 
in the two lowest 



Question 7 



The Hcntenee below htiH blank Hpacci*, each blank indicat- 
ing that a W(»rd Iiuh be^>n oinitl«Hl. lUMieaih the Hcntenee 
are live k'ttered hvIh of words. You are to rhctoHc the om? 
Het. of wonlH which« whi'.n inHortud in t\w Houtcnce, bent 
filH in with the meaning of the Hontenee aH a whole. 

From the Wrnt the iHhinder«i, <lcHpii(> an out want — , did 
what they could to — the ruthlcHH occupy inK power. 

(A) harmony . . aaHist 

(B) enmity .. cmbArraHH 

(C) rebellion . . foil 

(D) resistance . . dctitroy 
(K) actitiiescencc . . thwart 
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♦Corrocl iin.swcr 



This type of question is one of several commonly used in 
tests in which a measure of verbal aptitufle is sought. Spe- 
cifically, this type is designed to measure one aspect of 
reading comprehension: the s(ud(Mit's ability to recognize 
the consistency in logic and in stylo that i required of the 
elements in a sentence. If the stu<lent understands the 
implications of a sentence, ho should be able to select the 
answer that best fulfills the meaning of the sentence, The 
sentences deal with a wide variety of topics that the stu- 
dent is likely to have encountered in general reading. 

Examination of the question should indicate that the 
answer involves two words which are. in a sense* opposite 
in meaning, since the word **despite'' carries with it the 
implication that the islanders acted In one fashion* while 
presenting a somewhat different impression to the *'ruthless 
occupying power." With this relationship in mind, (A), 
(B), (C), and (D) can be eliminated .since all of those 
answers fail to give the sense of contrast that is required. 
If outward harmony existed, a.ssisting the occupying power 
would probably contribute to this state of harmony and 
strengthen it. If enmity existed, embarra.ssing the occupy- 
ing power would he one method of expressing this feeling. 
Should rebellion exist, then the islanders would be doing 
what they could to foil the occupying power: these terms 
do not imfily opposition. The same logic holds for **resist- 
ance , . dcvstroy." The correct an.swer, the only one imply- 
ing two opposed actions, is (E). 

In this analysis, 58 per cent of a group of college-bound 
secondary school students answered the question correctly, 
and it discriminated extremely well between the good stu- 
dents and the poor students. Performance on this question 
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7 



corrolatufl .60 wlih porformfuico on tJic tost fis n whole. 
The finnlysis shows that whc'r<»as nearly all of ihn inp oiio- 
fifth answered correctly, only 15 per cont of ihc lowest one* 
fifth did so. 



Questions 8*9 

The paHHiif^e below \h followod liy qiiestiotiH based on its 
oontent. After ruadiu^ the pnHSii^c, choOHe tlic beflt ani^wcr 
to each c|ucHtioti. AiiH%ver all (iiiesticMiH lullowin^ the pa.s- 
sujic on the buHis of what i.s Htntod or ituplit ?d in the paH»n^e. 

MiIPh e.slimale of Harriot's KJft.s niay have been 
extravagant. Then' contcniporarieH, few of wlxiiii knew 
hor well, were inclined to (ine.stion it. Ah Gold win 
Smith waspish ly remarked, **Miirs halhieination as 
(5) to hiH wife's j{ifts deprived hnn of all authority wher- 
ever that came in.'* Hut there can no longer be sub- 
stantial d(Hiht (if the ran}{e of her influence. Mr. Packo 
corroborates the conuiusion reached hy Professor 
ifayek in his ititrochiution to the Mill-Tayh»r corre- 

(10) spontlcnce — that Mill was only statin}^ sober fact 
when he credited Mrs. Taylor <who later became Mrs. 
Mill) with a lar^e shari; in the work published under 
his name. Nor was it a matter of feminine flattery or, 
as C'arlyle imagined, of ''those j{r€at dark eyes» that 

(15) were Hashing unutteralile things while he was* dis- 
coursing tho unntterabte.*' Mr. Paeke insists that, 
except for the j-oj^ic, the principles underlying the 
more important works of John Stuart Mill were de- 
fined although not actually composed by Harriet Taylor, 

(2«) and that every major woi-k after the I'tditical Koonomy 
was drafted or planned during their lirst few years of 
married life. This is not necessarily to accept Mill's 
view of his wife*s mental endc»wnientt4. She was obvious- 
ly a woman of lively mind and exceptional charm to 

(2!*) whoM} Mill responded intellectually as well as emo- 
tionally. Hut wc may still suspect that her ascendancy 
over him was, at bottc»m, one of those mysterious func- 
tions of personality rather than of intellect. 



Question 8 

According to Mr. Packc, Harriet's part in Mill's important 
works was 

(A) mainly restricted to small details 

(B) in the planning rather than in the execution of them 
<C) in the comfort of her personality and charm 

(D) only evident in the Logic 

(K) helping with the more difHcuit sections 
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Question 9 

This'p»ssaKe shows that CJoldwiii Smith's opinion of Mill 
was that Mill 

(A) was naturally the person who would know best about 
his own wife's intellectual powers 

(B) had no longer any authority over his wife 

(C) was able by some sixth sense to sec that hi« wife had 
outstanding intellectual ability 

(D) was very seriously mistaken about his wife's intellectual 
prowess 

{K) was not the real '^architect" of the writings published 
in his name 
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♦Correct answer 



Question 8 is fairly easy, and 58 per cent of a group of 
college -bound high school seniors answered it correctly. The 
question clearly ask'? for a restatement of Mr. Packers opin- 
ion as to tlie nature of Harriet Mill's contribution to her 
husband s work. In hnes 6-13, we learji that Packe corrobo- 
rates Professor Hayek and John Stuart Mill himself in the 
view that Mrs. Mill had a large share in the work pub- 
lished by Mill. In lines 16-22, we learn that Packe considers 
this contribution to consist of planning the work in outline. 
Accordingly, only (B) correctly states his opinion. Choice 
(A) is wrong because it suggests only a very small role for 
Mrs. Mill. Choice (C) fails to suggest any real intellectual 
contribution, which is counter to Packe's view that the prin- 
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ciplfs underlying tho works wrrr defincrl by Mrs. Mill. 
Choiro (D). which was the most popular choice of the 'east 
ablo fjroup, states that Mrs. Mill's contriliiition is only evi* 
(lent in the Logic, whereas tho passage reports that Packe 
specifically excopt(jfl the Lof^ic from his remarks. Finally, 
choice (E). while su.Cf?estinR an important role for Mrs. 
Mill, f^oes far heyon/l the pnssa^'e and describes her work 
as helping? when difTiculty arose. The pa.ssage nowhere siiR- 
fjests that she had such a role. The question correlates well 
with te.st score, as is evident from the int^reasinp projiortion 
of candidates who answer it successfully as the ability level 
increases. 

Question 9 proved to he more difllcult for the grouji (the 
two analyses aro derived from the same candidates). It is 
much more .specific, being based on the sentence in lines 
which quotes the remark of OoUhvin Smith. Smith, of 
course, is taking violent exception to Mill's opinion of his 
wife's gifts. He refers to this opinion as a "hallucination," 
and says that it dejirives Mill of all "authority." i.e.. all 
right to speak with authority, on that topic. 

Clearly, then, the correct restatement of Smith's opinion 
of Mill is (D). which says that Mill was mistaken about 
his wife. Choice (A) is directly opposite in meaning, and 
is clearly wrong. Choice (B) is wrong, for it misinterprets 
"deprived him of all authority wherever that came in." It 
is the most common mistake, anfl many quite able candi- 
dates made it. but it is clearly wrong. Choice (C) is wrong 
in the same sense in which (A) is wrong: it is too positive 
a thing to say. Finally, choice (E) is wrong because it is 
completely beyond the scope of Smith's remark, which had 
nothing to do with the question of who planned Mill's work. 

As in the first examjile. the question shows a definite 
tendency for the proportion of succe.ssful candidates to in- 
crease as the ability level rises. 



Questions 10-11 

Questions 10-11 arc based on the following German reading 
passage: 

Als in der -ersteii Naehkrie^szeit eine Flut von billif^er 
Sehundliteratur unser Land iibersehwemmtc — (in einem 
einzi^en Zurieher Sehulhaus fand man bei einer Durehsu- 
chunKtaUNondsoleher Heftehen) — bildete sieh zur Bekampf- 
un^ dieser Gafahr einc private Arbeitsgemeinsehaft. Da 
fiir Vcrbote keine gesetzliche Grundlaf^e vorhanden war, 
besehritt man den Weg, der sieh noeh immer als der beste 
bewahrt hat: man suehte das Sehltiehte durch das Gute 
aus dem Felde zu .sehlagen* Es sollte moraliseh und litera- 
riseh einwandfreie, spannende, lehrreiehe, zu ei^enem 
Tun anre^ende Ju^endlektiire 7,u beseheidenem Preis 
geschaffen werden. Da kein sehweizerioeher Verlag dafiir 
zu haben war, grundoten die Initianien im J ah re 1931 
einen eigenen Verlag. Pro Juventute iibernahm die Ge- 
sehaftsstelle, und die sehweizenHche Liihrersehaft setzte 
sieh bis ins letzte Bergdorf mit unermiidliehem £ifer fiir 
das Werk ein, das heute in der £rziehung der Sehweizer 
Jugend einen bcdcutcndcn Platz einnimmt. 

Translation: 

When our eountry was swamped by a flood of trashy litera- 
ture shortly after the war (a seareh of a single sehoolhouse 
in Zurich turned up a thousand eopies of sueh publications), 
a private cooperative was formed to combat this menace. 
Since there was no legal basis for suppressing these writings, 
the problem was approached in a way that has always 
proven to be the most sucecssful— ^that of using the good 
to rout the bad, Reading matter of unquestioned moral and 



literary value was to be made available to young people at 
modest prices, reading matter both .sii.spenseful and in- 
Htruetive, which woub* stimulate young people to inde- 
pendent action. Being unable to interest a Swiss publisliihg 
house in the project, the proiriiilcrs founded their own 
publishing houMr. Tro Juventute assumed management, 
while the entire teaching profession — down to the last 
mountain village — committed itself wholeheartedly to this 
undertaking, which now occupies a central position in the 
education of Switzerbunrs young people. 

Question JO 

Pro Juventute wurde gcgriindet, um 

(A) der schweizerisehen Lehrerschuft ein ICinkomnien zu 
verschafTcn 

(B) die Gesetze des Landcs zu umgehen 

(C) ausliindisehe Biieher voni Marktc zu vcrdrangcn 

(D) ijbersehwemniungcn des Landes zu verhiiten 

(K) wertvolle und preiswerte Jugendliteratur zu 
vcrofTcutlichcn 

Translation: 

Pro Juventute was founded in order to 

(A) provide income for the Swiss teaching profession 

(B) circumvent the laws of the land 

(C) squeeze foreign books from the market 

(D) prevent floods 

(E) publish worthwhile and moderately priced reading 
material for young people 
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♦Correct. an.swer 
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Question 11 

Man konntc die mindcrwerti^o Litoratur nicht, verbieton, 
well 

(A) Hie privati^s Ki^cntuin war 

(B) en kciiii' bossere Ju^cndloktiire ^Nib 

(C) aic nicht Resetzwidrl^ war 

(O) Hie billij? und woit v«rbrcitet war 

(K) sin dem deHchmaek dor Naohkric'^Hzeit entspraoh 

Translation: 

The traHhy litiiratiire could not be Bupprc»Hc<l bccaiiH(> 

(A) it was private property 

(B) no better reading material for youn^ people was avail- 
able 

(C) it was not illegal 

(D) it was cheap and widely road 

(K) it satisfied public taste in the postwar period 
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♦Correct answer 



This passage was included in a German test for college 
graduates applying for government service. A number of 
multiple-choice questions were based on this passage in 
order to ascertain whether the applicant comprehends w..th 
accuracy the factual information conveyed by this piece of 
straightforward newspaper reporting and whether he also 
understands the implications of important details as well as 
the author's purpose in writing this news story. The ability 
to understand such a highly idiomatic expository text was 
considered essential for the group for which the test was 
intended. 

In order to succeed on this kind of a test, the candidate 
needs a fair command of basic German vocabulary and of 
word structure and sentence structure^ as well as some ex- 
perience in reading contemporary German prose. He is not 
expected to know every word in the text, but rather to be 
able to infer the meaning of unfamiliar expressions from 
the context. 

From the group of comprehension questions based on 
this passage, two have been selected for detailed discussion 
in order to demonstrate the techniques used in good foreign- 
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lanfjimge testing, Basically, the questions are designed to 
give the examinee an opportunity to show wha^ ho can do 
with the knowledge and the skills he has act^uired. thus 
testinp this knowledge indirectly through its application to 
problems posed to him. 

Qiiostion 10 te.sts total comprehension of the passage. The 
question asked is, "For what purpose was Pro Jiivontut<' 
founded?" All of the five answer choices contain f?omr ele- 
ments derivod from the text. Init only the one correct an- 
swer is in complete agreement with the information offered 
in the passage. Those who understood certain words or 
phrases but failed to comprehend the ninin point were led 
lo choose one of the wrong answers. Tho statistical analysis 
shows that choice (A) attracted }3 cnndidatos. most of 
whom performed rather poorly on the tost as a whole. They 
vaguely understood that the teachers engaged in a business 
activity but inferred erroneously the teachers* motives for 
doing so. Choice (B) attracted a similarly poor group of 13 
candidates who based tl'iir responses on their recognition 
that the matter of legahty was discussed (lines 5-9), but 
who failed to understanfl its meaning in the context of tlie 
passage. Choice (C) is based on a knowledge of only a few 
scattered words or phrases: ''books,'* "market/' "no Swiss 
publishing house available." It attracted a few examinees 
who had only a piecemeal knowledge of vocabulary and a 
minimum of ability to comprehend meaning. A small group 
of examinees was led by the superficial connection through 
the word "flood" in the first line of the passage to pick (D). 
The correct choice (E) was selected by only one examinee 
of the lowest group and all 65 examinees in the top group. 
The percentage of those choosing (E) increases markedly 
with the ability of the sub-group. Hence the excellent cor- 
relation coefficient of .80. 

Question 11 deals with an important detail. Why was it 
impossible to suppress the trashy literature? Here again, 
each answer choice contains an element of information 
from the passage; but as a response to the question implied, 
only choice (C) makes sense. This question, with its un- 
usual correlation coefficient of .92, represents about the 
maximum discriminating power that can be attained with 
a single question. It is a strikingly effective measure of 
whether an individual can operate in German as effectively 
as in his native language. 



Question 12 

Direetions : Maintenant, vous allez entendre line conversa- 
tion entre deux personnes. Attendez la deuxieme replique 
et ensuite ehoisissez la rcponse qui convtent le micux. 

"Henriettc, passe-moi eette petite robe legcre qui se trouve 
dans mon armoirc a glaee.** 

**Attends un moment; j*ai la bouche p^eine dc pat-c denti- 
frice. J*aurai fini ma toilette dans un instant.** 

(A) Ne te presse pas. Je la ehereherai moi^meme. 

(B) Je l*ai trouvce tout & Thcurc. 

<C) Quand tu auras fini de te peigncr. 
(D) Qui, je Tai repass£*e hier soir. 
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* Correct answer 



The testing of a student's ability to understand a foreign 
language when spoken is a relatively recent development. 
Students listen to the recorded voices of native speakers in 
statements, short conversations, and short narrations. Sug- 
gested answers to questions based on the spoken material 
are printed in the students' test books. In listening com* 
prehension tests the foreign language is used throughout, 
not only in the test questions but also in the directions to~" 
students. The sample question given here was-part of a test 
designed for teachers of French in secondary schools. Para- 
phrased in English it reads: 

Directions: Now you will hear a conversation between two 
people. Wait for the reply of the second speaker and then 
select the reply which is moat appropriate. 

Recorded conversation: 

''Henrietta, hand mc the lit tic summer dress which is in my 
closet**' 

''Wait a moment; I have a mouth full of toothpaste. I shall 
be through in a minute.** 

Answer choices: 

(A) Don't hurry. I shall get it myself. 

(B) I found it just a little while ago. 

(C) When you are through doing your hair. 

(D) Yes, I ironed it last night. 

To select the right answer to this question, the student has 
to apply his knowledge of French grammar, vocabulary, 
and idiomatic expression, and his understanding of the 
French sound system. The students who have grasped the 
meaning of what they have heard will select (A) as the 
only choice that makes sense in the context. The literal use 
of the French word "trouver** <to find) in choice (B) mis- 
leads some who have not understood that the idiomatic ex- 
pression "se trouver" in tho conversation means simply 
*Ho be in a certain place." Choice (C) misleads those who 
have understood that the pf'cond woman will soon be fin- 
ished with v hstever she is doing but have not understood 
the nature of her activity. Choice (D) was selected by some 
stude.tts who mistook "passer** (to hand) in the conversa- 
tion for "repasser" (to iron). This question was answered 
correctly by 70 per cent of those who attempted it. All but 
one of th'^ top fifth selected the right answer, whereas only 
35 per cent of the lowest fifth did Ro. 



Question 13 

The concept of the plasma mcmbrHne ss a simple sievo-like 
structure is inadequate to explain the 

(A) passage of gases involved in respiration into and out 
of the cell 

(R) passage et* simple organic molecules such as glucose 
into the cell 

<C) failure of protein molecules to pass through the mem- 
brane 

(D) inability of the cell to use starch without prior 4iigestion 

(E) ability of the cell to admit selectively some inorganic 
ions while excluding others 
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♦Correct answer 



This question was asked of high school students who were 
completing a college-level course in general biology. The 
question was a relatively easy one for the group, as evi- 
denced by the fact that 80 per cent of the students selected 
the correct answer. The ability to handle this question suc- 
cessfully involves the knowledge that the living plasma 
membrane has properties in addition to those served by a 
piece of cellophane or similar material used in classical lab- 
oratory demonstrations of osmosis. A membrane such as 
cellophane can indeed be thought of as a porous structure. 
Only those molecules that are smaller than the largest of 
the pores would be able to pass through, accounting fully 
for the fact that both gases and glucose* whose molecules 
are relatively small, do pass through. The concept of the 
plasma membrane as a sieve also accounts adequately for 
the fact thai protein molecules cannot pass through, since 
protein mole^^ules are among the largest known. Choice (D) 
is a poor choice since it is well known that starch molecules 
are very large; as one might expect, only a few of the stu- 
dents in the sample selected this choice* Experiments have 
shown that certain ions of inorganic salts, acids, and bases* 
even though these ions are small, penetrate some plasma 
membranes very slowly, but pass readily through other 
plasma membranes having pores of the same size. If only 
the "sieve" nature of the membrane accounted for the selec- 
tive passage of materials through it, one woiild expect these 
ions to pass through all membranes rapidly* To account for 
the actual phenomenon, then, one must have recourse to 
nore elaborate interpretations, notably that electrical prop- 
erties of the living membrane selectively attract and repel 
certain ions and that the chemical nature of the membrane 
constituents selectively allows passage of materials that are 
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soluble in these constituents. Tho statistical analysis shows 
that better students selected (E) as Die only phenomenon 
not explainable by the conception of the plastnu membrane 
us a j»imp]c sieve. 



Qurstion 14 

Which of the Kraphn couhl represent ihc» velocity Him o re- 
lutioniihip of a biix that han no external frieticiti and whoio 
diak ia fastened aoUdly to it? 



Questions 14-15-16 

Quetitlona 14-16 relate to the following information and 
graph b; they consist of Ave lettered answer choices followed 
by a list of numbered questions. For each question seleet 
the one lettered answer whieh is most elosely related to it. 
An answer may be used once, more than onee, or not at all« 

A standard impulse is Ktven at time to to each of the several 
closed boxes having identical masses and identical ap- 
pearance* The impulse starts the box into motion on a 
horizontal plane. There may or may not be friction (referred 
to as ''external friction**) between any box and the plane. 
It is Icnown that each box contains a disk whose mass is 
equal to that of the box and which is restrng i>n the bottom 
of the inside of the box. There may or may not be friction 
(referred to as "internal friction'*) between any disk and 
the bottom of its box. The disks may rebound in a perfectly 
elastie way from the walls of the box ("elastic walla^'), or 
they may stick to the walls on colliding ("sticky walls^'). 

The grapht) below show the velocities of various boxes as 
a function of time. ' 
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(E) None of the above 
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•Correct n newer 



This set of questions appears in one of the achievement 
tests developed to accompany the high school physics 
course of the Physical Science Study Committee. 

Question 14 is the first of several questions based on the 
situation described. The series of questions requires a 
knowledge of certain aspects of kinematics and dynamics: 
an understanding of Newton's laws of motion and the con- 
cepts of velocity, acceleration, and momentum. The student 
must apply this knowledge to an unfamiliar problem and 
answer the questions in terms of graphs, a frequently used 
tool of the physical scientist. 

The situation described is an idealized one since the com- 
plications of friction are occasionally ignored. The study of 
phenomena as they would occur under certain idealized 
conditions is a major way to attack complex scientific 
problems. 

Question 14 pertains to a box with no external friction 
and with a disk fastened solidly to the bcx« When a stand- 
aid impulse is given to the box with its attached disk at 
time t»i the box would be set in motion. Since no friction 
or other force acts on the box after the initial impulse, the 
box would neither accelerate nor decelerate but would con- 
tinue in motion at constant velocity. Choice (A) illustrates 
this motion. 

The question was extremely easy; 342 of the 370 students 
(92 percent) who attempted it answered it correctly. None- 
theless, the correlation between performance on this ques- 
tion and the test as a whole is high (.56). 

The most discriminating incorrect answer choice was 
(B). Several students in the lowest fifth on the test seemed 
to recognize that in a frictionless situation, like that of this 
question, the velocity would not change with time, but they 
failed to recognize the simplicity of this situation and chose 
(B), where the velocity pattern is constant with time but 
where the velocity regularly drops to zero for short 
intervals. 



Question 15 



Question W 



Which of thu Kraphd could ruproAoni Iho vulocUy«liinu ro- 
latioaihip of thu NAinct boi with oilornal friction? 



Statist irnl Analysis 
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♦Correct answer 



A box with elaMtic wallM and no internal or external friction 
hati ilfl diMk initially rcHtin^ in the miilcllu of the box. Which 
of the MraphM could reprcHent the velocity-time relationnhip 
of thiM combination? 
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One condition is now added to the situation described in 
the preceding question. The student is asked to choose the 
graph which could represent the motion of the box with 
external friction. The force of friction would act to 
decelerate the box and the attached disk at a constant 
rate. Therefore, the velocity- time relationship of the box 
would be: 



•Correct answer 




Since none of the graphs given represents this motion, 
the answer is (E) , None of the above. 

This question was considerably more difficult than ques- 
tion 14; only 48 per cent of those who attempted it an- 
swered it correctly. One must determine what the velocity- 
time relationship would be and then recognize that the 
correct graphical relationship is not offered as a choice. 

The correlation between performance on the total test 
and performance on this question is very high, .70. (B) and 
(C) were the most common choices of students .with low 
scores on the test. These students apparently failed to 
realize that, as long as the disk is fastened to the box, the 
situation is as simple as if the disk were not present. The 
larger number of them chose (C), where the maximum ve- 
locity does drop with time, but both (B) and (C) have the 
weakness that they are appropriate only for more complex 
situations. 



When the standard impulse is given to the box described in 
this passage, the box moves until the disk inside, free to slip 
along the bottom since there is no friction, hits the trailing 
end of the box. At this point, there is an elastic collision 
between the box and the disk, which have equal masses. 
All the momentum of the box is transferred to the disk, and 
as a result, the box stops and the disk moves across the 
bottom of the box. The disk soon hits the forward end of 
the box in a second elastic collision. The momentum is once 
more transferred to the box, the disk stops, and the box 
once again moves at the same velocity as before. The ve- 
locity of the box is undiminished since there are no fric- 
tional forces acting to slow the motions. Choice (B) repre- 
sents this situation. This question was answered correctly 
by 57 per cent of the examinees. The correlation between 
performance on this question and performance on the total 
test in which it was used was high, .61. Only five of the 
students with the highest fifth of the test scores failed to 
analyze the situation correctly and to select the correct 
answer. Students with the lowest scores apparently did not 
understand the situation. Although very few of them omit- 
ted the question, their responses are distributed among 
choices (A), (B), (C), and (D) as if they were answering 
at random. Few of the students with the lowest scores chose 
the "None of the above" response, indicating their belief 
that the correct graph was presented, but they were, in 
general, not able to decide which one it was. 



ERIC 



13 



QuGfttion 17 

One mothod of olitainin»{ '*artiflcUl Kravlty** in • space 
station Js to haw thv Htation rotatinft about axis AA as it 
revolves around Karth. 



Side View 




The inhabitants of the space station would call which di- 
rection '*down*'? 

(A) Direction 1 

<B) Direction 2 

(C) Direction 3 

(D) Direction 4 

(E) Any one of the four, depending on speed of rotation 
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Per cent of total group of 170 studenta BiiBwerinfr correctly 62% 

Correlation between success on this question and total score on test ... 44 



* Correct answer 

This was part of a set of questions administered to high 
school students who were completing a year of high school 
physics. The question illustrates the kind of response that 
can be expected of M'ell-trained students of high school age 
when they are presented with a relatively novel situation 
which is based on fundamental concepts from the field of 
mechanics* 

This question requires that the student consider the na- 
ture of a possible mechanism for providing a "down*' direc- 
tion in a space station to simulate the gravitational "down*' 
so important in our normal activities on Earth. Choice (C) 
is the direction normally conRidered down in diagrams. 
Although this direction is not significant in the space sta- 
tion, a sizeable number of the poorer physics students 
chose it* Other students assumed that the "down'* direction 
would be that one toward the center of rotation of the sta- 
tion, choice (D)* However, objects free to move in the space 



station bohave as do particles in a centrtfugo and "fall" to 
tho outer edge. This direction, (B), then is the "down' 
direction in the rotating station. The other choices, (A), 
**up" an it is usually represented in diagrams, and <E), a 
direction whicn depends on the speed of rotation, were not 
selected by many students. 

Question 18 

Intensity of color may be used as a measure of concentra- 
tion. The permanganate ion MnOi* is intensely purple. If 
the following salts were separately dissolved in water to 
make 0.001 -molar solutions, the solution having the most 
intense color would be 

(A) KMnOi 

(B) NaMnOi 

(C) NaMn04 . 3HiO 

(D) Ca (MnOOs 

(E) AgMn04 
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Per cent of total group of 300 studenta answering correctly 28% 
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*Correct answer 

This question, which was asked of high school chemistry 
students, involves a rather simple concept with which all 
students at this level would be expected to be familiar. The 
concept might be stated as follows: Even though the num- 
ber of moles in a liter of solution of each of several salts is 
the same, the concentrations of the ions of these salts need 
not be the same. A mole of Ca(Mn04)2 provides two moles 
of MnO;^ ions while a mole of each of the other salts listed 
provides only one mole of MnO;^ ions. As a result, when the 
numbers of moles of the salts present in solution are the 
same, a solution of Ca(Mn04)2 contains more moles of 
MnO;' ions than any of the other solutions, and is therefore 
more intensely colored. 

The setting of the question is somewhat unusual, and the 
question is, therefore, useful in discrimindting between 
students who are able to see the application of a familiar 
concept in an unusual situation and those who are not. 
About 20 per cent of the students selected the correct re- 
sponse (D). Of the other compounds listed, the most famil- 
iar one is KMnO^, choice (A). A number of students ap- 
parently were unable to see the application of any concept 
in this situation and so selected the substance whose form- 
ula was most familiar, assuming that a familiar compound 
is most likely to be right. Nearly half of the lowest fifth of 



the students omitted the question, apparently because they 
did not realize that a familiar concept would lead to the 
answer. Students who chose (B), (C), and (E) demon- 
strated that they did not understand the moaning of 
"mole": (B) and (C) list the same salt, first in anhydrous 
form and then hyd rated; (E) lists a salt of a lieavy metal 
with a resulting high molecular weight. These difTerences 
among the salts do not affect the number of MnO^ ions 
introduced into the solution by a mole of each salt and 
thus choices (B), (C), and (E) are incorrect. 



Quegtton 19 

This question m based on the following situation: 

A piece of mineral is plaeetl in a bottle half-filled with a 
colorless liquid. A two-holcd rubber stopper is then placed 
in the bottle. The system is then sealed by inserting p 
thermometer and connecting a glass tube to the stoppered 
l)ottleanda beaker of limewater as shown in the accompany- 
ing diagram: 



I f ■ ■ 



thermometer 



colorless 
liquid 
niineroJ 
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The following series of observations is recorded: 

I* Observations during the first few minutes: 

1. Bubbles of a colorless gas rise to the top of the stoppered 
bottle from the mineral. 

2. Bubbles of colorless gas begin to come out of the glass 
tube and rise to the surface of the limewater. 

3. The limewater remains colorless throughout this period 
of time. 

4. The thermometer reads 20''C. 

II. Observations at the end of thirty minutes: 

1. Bubbles of colorless gas continue to rise in the stoppered 
bottle. 

2. The piece of mineral has become noticeably smaller. 

3. There is no apparent change in the level of the colorless 
liquid in the bottle. 

4. The colorless liquid in the bottle remains colorless. 

5. The thermometer reads 24°C. 

6. The limewater is cloudy. 

Which one of the following is the best explanation for the 
appearance of gas bubbles at the end of the tube in the 
beaker of limewater? 

<A) The pressure exerted by the colorless liquid is greater 
than that exerted by the limewater. 

(B) The bubbles coming ftom the mineral cause an in- 
creased gas pressure in the stoppered bottle. 

(C) The temperature increase at the end of thirty minutes 
causes an expansion of gas in the stoppered bottle. 

(D) The decrease in the size of the piece of mineral causes 
reduced pressure in the stoppered bottle. 

(£) The glass tube serves as a siphon for the flow of gas 
from the bottle to the beaker. 
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This question is ta]<on from a test designed to he u.sod with 
a new curriculum in high school chemistry. The question 
is only one of a series based on the experimental situation 
described. Questions in the series are grouped in sequence 
relating to the situation in order to permit the student to 
think intensively in one setting for an extended period of 
time. The student is asked to deal with a realistic labora- 
tory situation — one he has not yet encountered in the 
course at the time the test was given — and to employ scien- 
tific problem-solving ability in using the data given to an- 
swer the questions. 

Choice (A) is both vague and irrelevant It is unspecific 
about where the pressure is exerted and has nothing to do 
with the cause of the bubbles. 

Choice (C) sounds plausible. In itself, the statement is 
not incorrect, since the temperature increase will cause an 
increase in gas pressure in the stoppered bottle. However, 
students who are in command of the subject will realize 
that the increase in gas pressure due to the rise in temper- 
ature is insignificant compared with that caused by the 
bubbles coming from the mineral. 

Choice (D) embodies an incorrect statement. Some stu- 
dents may not realize this, but even if the statement were 
correct, it would offer an incorrect explanation for the ap- 
pearance of the bubbles in the beaker. 

Most of the good students rejected choice (E), but many 
less able students chose it, probably because the arrange- 
ment looks somewhat like a siphon and they had heard of 
the use of siphons in transferring fluids. 

Students who understand the forces at work in the situa- 
tion described will know that bubbles would appear at the 
end of the tube in the beaker when the pressure exerted 
by the gas from the stoppered bottle exceeded the pressure 
exerted by the limewater at the end of the tube. They will 
also realize that the limewater pressure would remain 
essentially constant. Choice (B), therefore, sets forth the 
condition that would account best for the appearance of 
gas bubbles at the end of the tube in the beaker. 
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Irnui Biirrounding Huch npliludo qu(>fttions in the need (o 
nx'oid familiar U*x(book mntrrinl while ai (he snme time 
rostrictinff the content level to thnt which woa mnRtercd 
enrlier in the junior hiRh school — Reometry nnd Rimple 
nlcebra. If thin need in met. then (he fociiR of (he aptitude 
lofit in whi«re it nhould >)o. namely on ingenuity in the roIv- 
ing of novel pmblemn. rather than on the recall of memo* 
rizoil proco<lure8. Thin ini;enuity. or what Home like to call 
"mathematical maturity." proluibly improves with takini: 
RtMid mathematics coursers throughout high school but is 
not much affected by short* range cramming. 

This question involves proce<lurcs which the student is 
extremely unlikely to have encountered in high school text* 
iKKiks: the content knowledge required is certainly not l>e* 
yond what is taught in the junior high school. The solution 
procecKls as follows: 

The lengtlis of the three sides of the squares are 10. 4» 
and 9. reading from left to right. Therefore, initially PQ 
equals 23. If the final length of PQ is to be 21. the amount 
of reduction of the side of the middle square must be 2. The 
middle square must consequently be reduced from an area 
of IG to nn area of 4; that is» it must bo reduced by 12 
square units. 
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Ciood mathematicH achievement questions h'ke (his one re- 
semble aptitude ti*st questions in requiring mathematical 
ingenuity and maturity for their solution. They also require 
Rpedflc content knowledge appropriate to the level of the 
test, in this case the Intermediate Mathematics Achieve* 
ment Test offered by the College Entrance Examination 
Board. 

This question is designed to exhibit the general relations 
lK*tween roots and coeflDcients in the quadratic equation. 
Able students with clear understandings of thesn relations 
will be able to solve this problem by inspection, thus saving 
time for other parts of the examination. Less able students 
may be able to solve it by more laborious methods. Students 
without any knowledge will usually miss or omit it. 

The solution depends on the important knowledge that 
in quadratic equations of the form x' 4- Bx + C « 0. C 
equals the product of the rooto and B equals the negative 
of the sum of the roots. By inspection, one can see that B 
is written as the negative of the sum of two quantities. The 

fact that they must bo the rootfl. a and — , is confirmed by 

a 



the fact that their product is C, or 1. A more laborious 
method in volves the use of the quadratic formula, 
- B ± VB^ - 4AC , 

— , where A is the coefficient of x2 and is 

2A I 

equal to 1, B is. equal to - (a + 7, ), and C is equal to 1, 

A third and also laborious method involves factoring the 
left member of the equation nnd setting each factor equal 
to zero. 

The discrimination power of the question can be observed 
by noting the ability level of the students selecting the cor- 
rect answer A progression of students choosing (A)» the 
right answer, can be seen from 8 in the lowe.st ability group 
to 37 in the highest ability group. Choice (C) attracted 47 
students, probably because of a mixup in sign in selecting 
the sum of the roots, There is some reason to believe that 
students who chose (B) used the quadratic formula above 
and failed to divide by 2 at the end. 
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Question 22 

The question below is followed by two statements, labeled 
(1) and (2), in which certain data are ^ivcn. In this question 
you do not actually have to compute an answer, but rather 
you have to decide whether the data given in the state- 
ments arc sufficient for answering the question. Using the 
data given in the statements plus your knowledge of mathe- 
maticM and everyday facts (such as the number of days in 
July), you are to select answer 

(A) if statement <1) ALONE is sufficient but statement (2) 
alone is not sufficient to answer the question asked, 

(B) if statement (2) ALONE is sufficient but statement (1) 
alone is not sufficient to answer the question asked, 

(C) if both statements (1) and (2) TOGETHER are suffi- 
cient to answer the question asked, but NEITHER 
statement ALONE is sufficient, 

(D) if EACH statement is sufficient by itself to answer the 
question asked, 

(E) if statement (1) and (2) TOGETHER are NOT sufficient 
to answer the question asked and additional data spe- 
cific to the problem are needed. 

If z is a whole number, is it a two-digit number? 
(1) x2 is a three-digit number. 
<2) lOx is a three-digit number. 



One of the abilities which has been receiving increasing em- 
p)iasis from the elementary school through college is that 
of judging the relevancy of data in the solution of problems 
in mathematics, science, and social studies. Measurement 
of the extent to which a student has developed this ability 
near the end of the high school years is believed to be im- 
portant in predicting scholastic success in college. To ac- 
complish such measurement the type of test question pre- 
sented here has been designed, validated by research, and 
is now in use in the College Entrance Examination Board s 
Scholastic Aptitude Test. This type of question is used in 
arithmetic, algebra, and geometry. It shifts the emphasis 
from rote, manipulative skills to higher level judgments and 
reasoning. 

This question shows the very great versatility of this 
type, for it requires little factual knowledge and the sim- 
plest aspects of elementary algebra^ but does require a con- 
siderable degree of numerical judgment* 

Fact 1 alone is sufficient because the square root of any 
three-digit square is a two-digit number. Fact 2 alone is 
sufficient because, whenever a three-digit multiple of 10 
is divided by 10, the result is a two-digit number. The cor- 
rect answer is therefore (D). 

This question and its analysis provide an interesting in- 
sight into the functioning of this type of question. When 
the correct answer is (D), there is a sense in which the 
question becomes two questions, since two separate suffi- 
ciencies must be determined. In this case, when the suffi- 
ciency of Fact 1 is considerably more difficult to determine 
than the sufficiency of Fact 2, a certain number of moder- 
ately able people can be expected to choose (B) as the 
answer. One of the reasons that may have made the suffi- 
ciency of Fact 1 difficult for them to see is that although 
the square root of any three-digit number is always a two- 
digit number, the square of a two-digit number is not nec- 
essarily a three-digit number* The latter point may have 
led some to conclude erroneously that Fact 1 was not suffi- 
cient. Fact 2, on the other hand, can more easily be seen as 
sufficient because only a two-digit number can be multiplied 
by 10 to produce a three-digit number The higher ability 
of the 121 selecting (D) is sufficient to give the question a 
good index of discrimination* The 20 choosing {C) appar- 
ently failed to realize that each of the two facts is inde- 
pendently sufficient* 
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